A Fast Greedy Algorithm for Generalized Column Subset Selection

This paper defines a generalized column subset selection problem which is concerned with the selection of a few columns from a source matrix A that best approximate the span of a target matrix B. The paper then proposes a fast greedy algorithm for solving this problem and draws connections to different problems that can be efficiently solved using the proposed algorithm. 1 Generalized Column Subset Selection The Column Subset Selection (CSS) problem can be generally defined as the selection of a few columns from a data matrix that best approximate its span [2–5,10,15]. We extend this definition to the generalized problem of selecting a few columns from a source matrix to approximate the span of a target matrix. The generalized CSS problem can be formally defined as follows: Problem 1 (Generalized Column Subset Selection) Given a source matrix A ∈ R, a target matrix B ∈ R and an integer l, find a subset of columns L from A such that |L| = l and L = argminS ‖B − P B‖F , where S is the set of the indices of the candidate columns from A, P (S) ∈ R is a projection matrix which projects the columns of B onto the span of the set S of columns, and L is the set of the indices of the selected columns from A. The CSS criterion F (S) = ‖B−P B‖F represents the sum of squared errors between the target matrix B and its rank-l approximation P B . In other words, it calculates the Frobenius norm of the residual matrix F = B − P B. Other types of matrix norms can also be used to quantify the reconstruction error [2, 3]. The present work, however, focuses on developing algorithms that minimize the Frobenius norm of the residual matrix. The projection matrix P (S) can be calculated as P (S) = A:S ( AT:SA:S −1 AT:S , where A:S is the sub-matrix of A which consists of the columns corresponding to S. It should be noted that if S is known, the term ( AT:SA:S −1 AT:SB is the closedform solution of least-squares problem T ∗ = argminT ‖B −A:ST ‖ 2 F . 2 A Fast Greedy Algorithm for Generalized CSS Problem 1 is a combinatorial optimization problem whose optimal solution can be obtained in O (